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Abstract 

Background: The Australian Government Department of Veterans' Affairs (DVA) funds an ongoing health 
promotion based program to improve use of medicines and related health services, which implements 
interventions that include audit and feedback in the form of patient-specific feedback generated from administrative 
claims records. We aimed to determine changes in medicine use as a result of the program. 

Methods: The program provides targeted patient-specific feedback to medical practitioners. The feedback is supported 
with educational material developed by a clinical panel, subject to peer review and overseen by a national editorial 
committee. Veterans who meet target criteria also receive educational brochures. The program is supported by a 
national call centre and ongoing national consultation. Segmented regression analyses (interrupted time series) 
were undertaken to assess changes in medication use in targeted veterans pre and post each intervention. 

Results: 12 interventions were included; three to increase medicine use, seven which aimed to reduce use, and 
two which had combination of messages to change use. All programs that aimed to increase medicine use were 
effective, with relative effect sizes at the time of the intervention ranging from 1% to 8%. Mixed results were seen 
with programs aiming to reduce inappropriate medicine use. Highly specific programs were effective, with relative 
effect sizes at the time of the intervention of 10% decline in use of NSAIDs in high risk groups and 14% decline in 
use of antipsychotics in dementia. Interventions targeting combinations of medicines, including medicine 
interactions and potentially inappropriate medicines in the elderly did not change practice significantly. 
Interventions with combinations of messages targeting multiple components of practice had an impact on one 
component, but not all components targeted. 

Conclusions: The Veterans' MATES program showed positive practice change over time, with interventions 
increasing use of appropriate medicines where under-use was evident and reduced use of inappropriate medicines 
when single medicines were targeted. Combinations of messages were less effective, suggesting specific messages 
focusing on single medicines are required to maximise effect. The program provides a model that could be 
replicated in other settings. 
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Background 

The difficulty in translating research findings to the 
practice setting has been well described [1,2], particu- 
larly in the area of medicines use. Under-prescribing of 
effective medicines is common, as is use of too much 
medicine, use of the wrong drug, and use of an inappro- 
priate medicine regimen [3,4]. Bridging the evidence- 
practice gap in the area of medicines-related health care 
is critical because medicines are the most commonly 
used health care intervention [5], and there is a significant 
gap between existing evidence and practice in relation to 
medicine use [6]. Implementation research provides evi- 
dence to guide the design of quality improvement pro- 
grams, which can promote the translation of research 
findings to practice and improve the use of medicines, pa- 
tient care, and health outcomes [1]. 

Audit and feedback are often promoted to improve 
use of medicines; however, systematic reviews of inter- 
ventions to improve uptake of evidence in practice have 
found mixed success [1,7]. A systematic review of ran- 
domized controlled trials of audit and feedback found 
that some interventions were associated with a large in- 
crease in adherence to practice guidelines, while others 
had a negative effect [7]; the median relative percentage 
practice improvement was 8 percent, with the adjusted 
risk ratio varying from 0.7 to 18 across studies [7]. The 
duration of effect following intervention also varied, with 
improvements in practice seen in some studies at up to 
six months follow-up, while in other studies there was 
no difference between intervention and control groups 
at three weeks post intervention [7]. Some studies found 
no difference between intervention and control groups 
at any point of time during follow-up [7]. Audit and 
feedback interventions were more likely to have an effect 
when baseline adherence to the targeted treatment 
guideline was low, however, the effect was still modest 
[7]. This may be because many implementation studies 
do not consider communication, behavioral, and health 
promotion theories in their design. One review of guide- 
line implementation studies found that less than 10 
percent of studies identified the theoretical rationale 
underpinning the intervention [1]. While results vary, 
collectively, the evidence suggests audit and feedback is 
effective. From an implementation science perspective, 
the next research question becomes how to implement 
audit and feedback routinely as part of ongoing routine 
health-care improvement. 

In this paper we report the results of an ongoing 
health promotion-based quality improvement program 
that uses audit and feedback in the form of patient spe- 
cific feedback generated from administrative claims data 
to improve use of medicines in the elderly Australian 
war veteran population. Since 2004, the Australian Govern- 
ment Department of Veterans' Affairs (DVA) has funded a 



quality improvement program, the Veterans' Medicines 
Advice and Therapeutics Education Services (Veterans' 
MATES) program [8,9], to bridge the evidence -practice 
gap in the provision of health care to Australian war vet- 
erans. The overall aim of this paper is to determine 
changes in medicine use as a result of the program. 

Methods 

The Veterans' MATES program aims to improve medica- 
tion use and health outcomes for all persons in the vet- 
eran community by delivering interventions to general 
practitioners (GPs), pharmacists, and veterans. Social cog- 
nitive theory [10,11], the transtheoretical model [12,13], 
and the health promotion model Precede-Proceed [14] 
were used as the theoretical frameworks that underpinned 
the program and were used to predict learning and behav- 
iour change (Figure 1). Key features of the implementation 
design included the ability to provide routine periodic in- 
terventions with ongoing evaluation suitable for participa- 
tion by all practitioners (Figure 1). The main intervention 
was patient-specific feedback that identified for GPs their 
patients with potential medication-related problems. The 
feedback included a list of the patient's relevant medicines, 
contained notes identifying the potential problems, and 
included tick boxes for GPs to indicate the actions they 
would take in response to the information, including the 
need for a review of the patient's therapy (Figure 2). 
Supportive educational material was also provided that 
included advice to assist the GP in resolving the potential 
medication-related problem. In addition, the veterans 
identified in the GP mailing were mailed an educational 
brochure highlighting potential medication issues related 
to the topic and encouraging the veterans to speak with 
their doctor. The same educational material was also 
provided to all pharmacies and accredited pharmacists 
to enable pharmacists to support the practice change 
(Figure 2). The program was implemented four times per 
year. Mailings were delivered only to those GPs and veterans 
who met the criteria for each program and to all pharma- 
cies and accredited pharmacists. Between November 2004 
and September 2008, 16 educational interventions were 
distributed. Twelve of these interventions focused on 
changing medication use and are the subject this paper. 
The aims of the twelve interventions are listed in Table 
one. Four other interventions focused on health service 
use, including home medicines review and dose adminis- 
tration aids, and are not reported here. For the interven- 
tions, material was mailed first to GPs and pharmacists, 
with the educational material to veterans provided one 
month later to enable GPs to read the material prior to 
the veteran visit. 

In keeping with health promotion principles involving 
needs assessments and identifying barriers and enablers 
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Step 5: administrative 
and policy assessment 



Health education; Policy, 
regulation, organisation. 
Each intervention 
includes: 
Therapeutic Brief (GPs & 
pharmacists): Developed 
by a medical writer, 
supported by a clinical 
reference group, peer 
reviewed. Endorsed by 
national representative 
committee 

Patient specific feedback: 

Prescribing feedback to 
the GP, identifying 
patients who meet the 
criteria for the 
intervention. Includes 
patient review questions 
and response forms with 
call to action questions 
Veteran brochure: 
relevant to the theme of 
the intervention, 
highlights medication 
issues, encourages them 
to speak with their doctor. 



Step 4: educational and 
ecological assessment 



Predisposingfactors: 

Identified through 
stakeholder 

consultation. Addressed 
in therapeutic brief. 



v 



Reinforcing factors: 

Identified through 
stakeholder 

consultation. Addressed 
in therapeutic brief, 
prescriber feedback and 
response forms, 
organizational 
messages, peer 
messages, and national 
call centre suddoiI. 



A 



Enabling factors: 

Identified through 
stakeholder consultation. 
Addressed in therapeutic 
brief, prescriber 
feedback, patient review 
questions, call to action 
questions, organizational 
support and national call 
centre support. 



Step 7: process evaluation 



Step 3: Behavioural and 
environmental assessment 



Behavioural targets: defined 
prior to the intervention 
based on the known problem 
identified by social and 
epidemiological assessment 
and clinical reference group 
that meets monthly e.g. 
increase/decrease medicine 
use, increase testing. 



Environmental targets: 

Organizational 
stakeholders. 
Government, Health 
professional and consumer 
organizations all included 
in the intervention 



Step 8: impact evaluation 



Step 2: Epidemiological 
assessment 



Medication related 
— ^ problem analysis: 

Conducted quarterly to 
identify topics suitable for 
intervention. 

Uses administrative health 
claims data. 

Topics must be amenable to 
change by intervention 
Problematic in the veteran 
community 

Topics suitable for repeat 
messages over time are 
chosen. 



Step 1 : Social assessment 



Needs assessment: 

Consultations with 
stakeholders through 
reference groups for 
consumers and health 
practitioners (twice 
yearly) and national 
rep resentative 
committee (quarterly). 
Allows stakeholders to 
set priorities for topics. 



^. Step 9: outcome evaluation 



Figure 1 Components of veterans' MATES interventions, linked to steps in the precede-proceed model. 



to change, the program began with nationwide consultation 
to stakeholder groups, including health professional and 
consumer organizations (Figure 1). The program also 
established practitioner and consumer reference groups 
which meet twice a year to provide advice on topics, inter- 
ventions, and practitioner and consumer needs. The educa- 
tional material is developed by a medical writer, supported 
by a clinical reference group that meets monthly. Prior to 
publication, the material is peer reviewed and also reviewed 
by a national representative editorial committee to ensure 
both the quality of the material and stakeholder support 
for the material and intervention. The program is further 
supported by a national call centre staffed by clinical phar- 
macists who provide educational support, clarify issues, 
ensure problems are quickly identified, and accept com- 
plaints. National consultation with stakeholder organiza- 
tions and advice from veteran and practitioner reference 
groups is ongoing (Figure 1). 

Each educational intervention is based on a specific 
medicine or health-related topic. To identify topics, the 
prevalence of medication-related problems is determined 
using DVA administrative health claims data (Figure 1). 
In 2004 when the program commenced, there were ap- 
proximately 320,000 Australian war veterans, war widows, 
or widowers. Their median age was 80 years; over 40 
percent regularly used two or more medicines, and 13 



percent regularly used five or more medicines. With vet- 
erans aged 70 years and over dispensed an average of 45 
prescriptions per year, the population is at high risk of 
medication-related problems because of their high medi- 
cation use. DVA health claims data contain details of all 
DVA subsidized prescription medicines, medical and al- 
lied health services, and hospitalizations for veterans. 
Veterans are entitled to receive all medicines subsidized 
on Australia's national pharmaceutical benefits scheme 
and entitled to receive additional medicines that are subsi- 
dized under the Repatriation Pharmaceutical Benefits 
Scheme. DVA maintains a client file, which includes data 
on gender, date of birth, date of death, and family status. 
Medicines are coded in the data set according to the World 
Health Organization (WHO) anatomical and therapeutic 
chemical (ATC) classification [15] and the Schedule of 
Pharmaceutical Benefits item codes [16]. Hospitalizations 
are coded according to the WHO International Classifica- 
tion of Diseases [17]. 

The medication-related problem analyses were gener- 
ally based within the therapeutic areas outlined in the 
Australian government National Health Priority Areas 
(asthma, cardiovascular health, diabetes, mental health, 
and arthritis and musculoskeletal conditions) [18]. Criteria 
for each topic were that it was problematic in the veteran 
community, specific to medication or health service 
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Topic 26: Urinary incontinence 



Patient selection criteria: Dispensed a n>edicir 


le for urinary incontinence and 


or supplied with a continence product. 


Information included: Dispensed medicines as 
September 20 1 0 to 31 st December 201 0. 


»sociated with urinary incontine 


nee in the four month period 1st 



John Smith WESTMEAD SA 5000 

Baseline (1 September 2010 to 31 December 2010) 



I Drug Name 


Brand 


I Strength 


I Last Dispensed 


Other Prescriber 


*OXYBUTYN I N HCL 


Ditropan 


Tab 5mg 


19/04/2010 


N 


**TRAMADOL HCL 


APO-Tramadol SR 


Tab 150mg (SR) 


06/09/2010 


N 


"FRUSEMIDE 


GenRx Frusemide 


Tab 40mg 


07/1072010 


Y 



What is the type of accomnxxlation: Community 

Date of the last medication review claimed: None claimed in last 12 months. 

Notes: 

•Medicine indicated for urge incontinence onry 

**tf incontinence symptoms followed medicine initiation, consider dose reduction alternative therapy or 
cessation 

Consider a medicines review 



Your action... 


□ 


Initiate patient review 


□ 


Change or cease medicine(s) 


□ 


Initiate medicines review 



Generated Trom veterans' MATES website (07KJ4/2011 09:37 &M] 

Tins report contains corndenttai pacent information ana is a partial recotd only 



Australian Government 
Department of Veterans' Affair* 



A Veterans'MATES 



1 ^Therapeuti 
[ Brief 



The impact of commonly used 
medicines on urinary incontinence 

Approximately 25,000 veterans are affected by urinary 
incontinence (Ul); of these, nearly two thirds are female and 
93% are aged over 75 years. 1 



Urinary incontinence adversely affects 
quality of life and it has been reported 
that men with urinary incontinence have 
poorer self-percerved general health than 
men who are continent. 2 In frail older 
persons it is associated with increased 
burdens on iheir carers which can 
precipitate admission to an aged care 
facility. Urge incontinence is associated 
with an increased risk of falls. 



The impact of medicines in 
precipitating or worsening Ul may be 
underestimated. Many commonly 
used medicines can adversely impact 
r. approximately 90% of 
• with Ul are dispensed at least 
one of these medicines and 36% are 
dispensed three or more.' The risk of 
incontinence increases as the veteran 
is dispensed more of these medicines 
(increasing the 'drug burden' or load ). 
They include verapamil, diltiazem. 
temazepam and ACE in 



Any medicine which acts on 
adrenergic or cholinergic 
receptors, or which affects 
cognition may impact on 
urinary continence. 



This Veterans' MATES therapeutic 
brief aims to increase awareness of 
commonly used medicines which 
can precipitate or worsen urinary 
incontinence. Whilst it may not be 
possible to cease these medicines, 
for those veterans where the 'load' 
of these medicines is significant. GPs 
may be able to improve incontinence 
symptoms by reducing dosages. 
This brief also covers assessment of 
urinary incontinence and strategies for 
treatment. 

Aetiology of urinary 
incontinence 3 ' 4 - 5 

Normal bladder function results from 
a complicated series of central and 
peripheral nerve signals. 

Several physiological changes occur in 
the lower urinary tract with increasing 
age. These can predispose to urinary 
incontinence and include: increased 
prevalence of involuntary detrusor 
contractions, a decrease in urethral 
closure pressures in women and 
the development of benign prostatic 
hypertrophy (BPH) in more than half of 



Inside 

U Aetiology of urinary incontinence 
tj Assessment of urinary incontinence 
O Medicines and urinary incontinence 
O Treatment of urinary incontinence 
O Further information 




Medicines (see table 1) 
Cognitive impairment 
Mobility (inability to get to 



Bladder and sphincter function 
Unnary tract infection 



-<s>— 



Figure 2 Intervention materials provided for the veterans' MATES program. 
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management, amenable to change by the interventions 
employed by the Veterans' MATES program, and able 
to be evaluated using DVA administrative data sets. In 
addition, the topics chosen were suitable for repeat mes- 
sages over time, to increase the persuasiveness of the 
messages [19]. In developing the intervention, strategies 
consistent with behavioural theories were established. 
Every educational intervention included strategies to raise 
awareness, improve knowledge, and encourage trial of ap- 
propriate behaviours. Each intervention included objectives 



to provide useful information to participants; to increase 
GPs' knowledge of the veterans they treat who meet the 
target criteria for the intervention; and to change medica- 
tion or health service use in line with the messages of the 
educational intervention. GPs and pharmacists could 
claim professional development points for participating. 
This provided an incentive for GPs and pharmacists as 
professional development reports are required for main- 
taining practicing certificates. Table 1 provides the final 
list of interventions and target groups. 



Table 1 Educational intervention topics and target audience 



Intervention 
number and topic 


GPs (n) 


Veterans (n) 


Pharmacists (n) 


Aim of educational intervention 


Target criteria for veterans mailed to: 


Interventions to increase use of medicines 






2: Beta-blockers, 
take the next step 
for heart failure 


6954 


12047 


N/A 


To increase beta-blocker use in 
veterans with heart failure 


Veterans dispensed medicines indicative 
of heart failure who were not dispensed 
a heart failure specific beta blocker 


3: Diabetes triple 
check 


8573 


16612 


5459 


To increase use of adjunct 
cardiovascular medicines in 
veterans with diabetes 


Veterans dispensed medicines indicative of 
diabetes (insulin and/or oral 
hypoglycaemics) 


15: Osteoporosis 


16876 


83110 


7967 


To increase uptake rates of bone 
density tests, and osteoporosis 
treatments 


Women aged 70-79, men aged 80-85 and 
those over 50 years admitted to hospital 
with a fracture from a same level fall 


Interventions to reduce use of medicines 








4: Clinical risk 
management: 
NSAIDs 


11242 


9885 


5447 


To reduce NSAID use in veterans 
with heart failure and diabetes 


Veterans with dispensed medicines 
indicative of diabetes and/or heart failure 


5: Antidepressants: 
three steps 
towards safer use 


12482 


42196 


5447 


To reduce potentially avoidable 
antidepressant interactions and 
duplicate therapy 


Veterans dispensed antidepressants 


6: Inhaled 
respiratory 
medicines: 
optimising use 


10720 


28670 


5447 


To decrease use of multiple devices 


Veterans dispensed inhaled respiratory 
medicines 


7: PPIs in GORD: 
Reduce the dose - 
keep the benefits 


13684 


62460 


5447 


To encourage use of lower-strength 
PPIs for maintenance therapy 


Veterans dispensed proton pump inhibitors 


8: Reducing 
adverse drug 
events for your 
veteran patients 


11050 


32484 


7074 


To reduce use of potentially 
inappropriate medicines in the 
elderly (aged over 70 years) 


Veterans 70 years of age or over, dispensed 
medicines that should be used with caution 
in the elderly (according to Beers [33] and 
McLeod [34] criteria) 


12: Antipsychotics 
in dementia 


3884 


6690* 


8089 


To reduce antipsychotic use for 
behavioural and psychological 
symptoms of dementia 
(in those aged over 65 years). 


Veterans aged >65 years dispensed oral 
antipsychotic medicines. 


14: COPD 


8785 


18096 


7880 


To reduce nebuliser use and reduce 
multiple device use 


Veterans dispensed tiotropium or 
ipratropium 


Interventions with a combination of messages 






10: Constipation: a 
quality of life issue 
for veteran patients 


9825 


29231 


7327 


To improve use of medicines for 
constipation; specifically, to increase 
the use of osmotic and bulk laxatives 
and reduce the use of contact laxatives. 


Veterans dispensed laxatives 


13: Clopidogrel 


8279 


16867 


7970 


To increase use of clopidogrel with 
aspirin and reduce concurrent use 
of clopidogrel with NSAIDs 


Veterans dispensed clopidogrel 



^Material was not mailed to veterans. For this topic, materials were provided to the GPs treating these 6990 patients to pass on to the patient if appropriate. 
NSAID, non-steroidal anti-inflammatory drug; PPI, proton pump inhibitor; GORD, gastro-oesophageal reflux disease; COPD, chronic obstructive pulmonary disease. 
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Evaluation 

Stakeholder satisfaction was evaluated for all interven- 
tions using a one-page reply paid response form that 
was mailed with the educational materials. The evalu- 
ation forms were designed to minimize additional work- 
loads for busy health professionals, thus, the forms were 
limited to no more than nine questions (one page) and 
all answers required a "tick-box" response only. Evalu- 
ation forms were created for every intervention with 
questions relevant to the targeted topic. Specific forms 
were created for GPs, pharmacists, and veterans. All tar- 
geted participants were mailed the evaluation forms. No 
follow-up reminders were sent. The response forms in- 
cluded questions on the usefulness of the educational 
material, as well as likely actions to be taken as a result 
of the program. Three months after each intervention, 
responses were collated. Individuals could only respond 
once for each intervention but could respond to multiple 
interventions if they had been targeted in more than one 
intervention. 

Statistical analyses 

Segmented regression analysis was used to estimate the 
effect of each intervention on medicine utilisation taking 
into account the baseline trend prior to the intervention 
[20]. A log-binomial generalised estimating equation was 
used with an ar (1) error structure to account for correl- 
ation between months, clustered by patient. The models 
included a constant term, a term for baseline trend prior 
to the intervention, an indicator term to estimate the 
change in level at the time of the intervention and a 
post-intervention trend term to determine the sustain- 
ability of the intervention over time. We calculated the 
number of people estimated to have changed behaviour 
as a result of each intervention based on the number of 
veterans targeted at the time of the intervention, the 
level change at the time of the intervention plus the 
monthly trend change at the time after the intervention 
compared to the pre-intervention trend over the 
24 month period. The time periods used for the analysis 
were as follows: the pre intervention period was the 
24 months prior to the month of mail out for each inter- 
vention, the intervention period was the four months 
immediately following the month of mail out, and the 
post-intervention period was the subsequent 20 months 
(i.e. month 5 to 24 post mail-out). All analyses were 
undertaken using SAS for windows, V9.1.3 SP4 (SAS In- 
stitute, Cary, North Carolina, USA). 

Ethics statement 

Ethics approval for the study was obtained from the De- 
partment of Veterans' Affairs Human Research Ethics Com- 
mittee (reference number: E004/016) and the University of 



South Australia Human Research Ethics Committee 
(reference number: P203/04). 

Results 

The twelve interventions implemented had an average 
target group size for each intervention of 33,000 vet- 
erans, 10,000 doctors, and 8500 pharmacies/accredited 
pharmacists. The target group size and aim of each indi- 
vidual intervention is listed in Table 1. 

Of the three interventions that aimed to increase medi- 
cine use, all achieved statistically significant increases at 
the time of the intervention. However, all post-intervention 
trends, while remaining positive, were lower than the pre- 
intervention trend (Table 2 and Figure 3). Estimated num- 
bers of patients with sustained behaviour change over the 
subsequent 24 months as a result of the intervention 
ranged from 642 to 3234 per intervention. 

Of the seven interventions that aimed to reduce use, the 
three that targeted one specific medicine to be reduced, 
NSAID use in those with heart failure or diabetes, high 
dose proton pump inhibitor (PPI) use, and antipsychotic 
use in those aged over 65, all achieved significant level 
changes at the time of the intervention, ranging from 10% 
to 14% (Table 3 and Figure 4). For proton pump inhibitors 
and antipsychotics, the impact was increased over the 
20 month follow-up period as evidenced by the statisti- 
cally significant further reduction in the trends post inter- 
vention compared to pre-intervention trends (Table 3). 
The estimated number of patients with sustained behav- 
iour change over 24 months ranged from 780 to 1725 for 
the three interventions. By comparison, the four interven- 
tions to reduce use that had multiple messages, medicine 
interactions (antidepressant interactions), potentially in- 
appropriate medicines in those aged over 70 and re- 
spiratory device use did not show significant effects. 
Pre-intervention trends, which in most cases were fall- 
ing, continued to fall. Duplicate antidepressant use was 
the exception, where usage increased despite messages 
that duplicate use was not appropriate. The two inter- 
ventions that targeted multiple respiratory device use 
for those on four or more different devices only reduced 
post intervention, but not at the time of the interven- 
tion (Table 3). 

Table 4 demonstrates the results of two interventions 
that had combined messages to increase one aspect of 
medicine use and decrease another. Significant changes 
were seen in keeping with intervention messages for clo- 
pidogrel at the time of the intervention with increased 
aspirin use and reduced NSAID use; subsequent trends 
remained unchanged. By contrast, laxative use did not 
change at the time of the intervention, but subsequent 
trends were statistically significant in keeping with inter- 
vention messages. 



Table 2 Impact of interventions to increase medicine use 



Intervention number and topic 
(number of veterans targeted) 


Targeted therapy 


Proportion of veterans 
on targeted therapy 
at baseline (%) 


24 month pre- 
intervention trend 


Intervention 
effect (%) 


20 month post intervention trend 


Estimated number with 
changed behaviour from 
intervention; sustained over 
two years post intervention 


2: Beta-blockers, take the next step 
for heart failure (n = 12047) 


Increase beta- 
blocker use 


8.7% 


Trend increasing 
1 .7% per month 


7% (p< 0.0001) 


Trend still increasing but reduced to 
0.6% per month (p < 0.0001) 


642 


3: Diabetes triple check (n = 16612) 


Increase ACE 
inhibitor use 


61.3% 


Trend increasing 
0.3% per month 


1.3% (p< 0.0001) 


Trend still increasing but reduced to 
0.1% per month (p< 0.0001) 


543 




Increase lipid 
lowering therapy 
use 


47.9% 


Trend increasing 
0.6% per month 


1.9% (p< 0.0001) 


Trend still increasing but reduced to 
0.5% per month (p = 0.015) 


1573 




Increase antiplatelet 
use 


41.9% 


Trend increasing 
0.4% per month 


3% (p< 0.0001) 


Trend still increasing but reduced to 
0.1% per month (p < 0. 0.0001) 


543 


15: Osteoporosis (n = 83110) 


Increase use in 
women 


9.1% 


Trend increasing 
1.1% per month 


2.8% (p < 0.004) 


Trend still increasing but reduced to 
0.4% per month (p < 0.0001) 


712 




Increase use in men 


4.7% 


Trend increasing 
1 .5% per month 


8.3% (p< 0.0001) 


Trend still increasing but reduced to 
0.2% per month (p< 0.0001) 


2522 



ACE inhibitor, angiotensin converting enzyme inhibitor. 
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2: Beta blockers, take the next step for heart failure-Rate of use of 
beta blockers by those with heart failure 

180 -1 

160 — 




« 40 
cc 

20 



0 H — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i 

^— Pre-intervention period: Mar 2003- Feb 2005. Monthly change: 1.017 (1.015-1.019) p<0.0001. This means that 
the number of veterans with CHF using a beta blocker was increasing by 1 .7% per month pre-intervention. 

^—Intervention period: Mar 2005 - Jun 2005. Change in use: 1.07 (1.04-1.10) p<0.0001. This means that the 
number of veterans with CHF using beta blockers increased by 7% at the time of the intervention. 

Post-intervention period: Jul 2005 - Mar 2007. Monthly change (relative to pre-intervention): 0.989 (0.986- 
0.991) p<0.0001. 

This means that the number of veterans with CHF using beta blockers was decreasing by 1 .1% per month 
compared to the pre-intervention trend i.e. overall, post-intervention the number of veterans with CHF 
dispensed beta blockers increased by 0.6% each mont 

Overall effect: 642 additional veterans with CHF were dispensed beta blockers as a result of the intervention 
Figure 3 Example of an intervention aiming to increase medicine use. 



The stakeholder evaluations for each intervention had 
a median response rate of 8% (interquartile range 7%- 
10%) for GPs, 9% (7%-9%) for pharmacists and 26% (24%- 
32%) for veterans. On average, 80% of GPs and 93% of 
pharmacists who completed the stakeholder evaluation 
found the interventions useful or very useful (Table 5). On 
average, 80% of veterans who completed the stakeholder 
evaluation found the information provided helpful or very 
helpful (Table 5). 

Discussion 

In our study, all educational interventions aiming to in- 
crease medicine use were effective, with relative effect 
sizes ranging from 1% to 8% at the time of the interven- 
tion. However, all of the interventions also had a statisti- 
cally significant effect on the post-intervention trend 
which was reduced compared to the pre-intervention 
trend. The reduced trend observed after the intervention 
could be due to the reduced naive patient pool which re- 
sults as a direct consequence of the immediate impact of 
the intervention. It also suggests that the effect of inter- 
ventions to increase use of medicines may not be sus- 
tained and would require repeat messages over time to 
be sustained. 

Educational interventions aiming to reduce unnecessary 
use of medicines had mixed results. Those with very specific 
messages (reduce use of non-steroidal anti-inflammatory 



agents in high risk patients; reduce use of high dose proton 
pump inhibitors; and reduce use of risperidone for the be- 
havioural and psychological symptoms of dementia) were 
all effective, with relative effect sizes up to 14%. Educa- 
tional interventions with more generic adverse event mes- 
sages (avoid antidepressant interactions, reduce potentially 
inappropriate medicine use in the elderly) had no measur- 
able effect on practice. Interventions less clearly linked to 
patient harm or poor health outcomes and with combina- 
tions of messages, such as those to reduce use of multiple 
inhaler devices, and improve the use of medicines for con- 
stipation were associated with practice change in some 
but not all aspects of care and were often found to be sta- 
tistically significant in the post-intervention trend only, 
not at the time of the intervention. It is likely that the 
combination of messages meant these interventions were 
less effective. The evaluation of these interventions may 
also have been confounded by patient practice, where 
these medicines may be used "as required" and thus 
change in dispensing quantities may be delayed, hence the 
positive effect in the post-intervention trend only. 

The relative effect of our results, which ranged between 
1% and 14%, is consistent with the median relative effect 
reported in the Cochrane review of audit and feedback of 
8% 7 . Our results suggest well implemented, behaviourally 
grounded, ongoing programs do achieve similar results to 
randomised control results. 



Table 3 Impact of interventions to reduce use 



Intervention number and topic 
(number of veterans targeted) 


Targeted therapy 


Proportion of 
veterans on 
targeted therapy 
at baseline (%) 


24 month pre- 
intervention trend 


Intervention 
effect (%) 


20 month post 
intervention trend 


Estimated number with 
changed behaviour as a 
result of intervention; 

sustained over two 
years post intervention 


4: Clinical risk management: 
NSAIDs (n = 9885) 


n 1 . Mf a in _ _ * 

Reduce NSAID use in 
those with heart failure 


15.6% 


Trend decreasing - 
0.8% per month 


-1 1.4% (p < 0.0001) 


Trend still decreasing 
(non-significant to prior) - 0.7% 
per month (p = 0.87) 


1 163 (680 heart failure patients 
and 483 diabetes patients) 




Reduce NSAID use in 
those with diabetes 


1 7.9% 


Trend decreasing - 
1 .2% per month 


-10.2% (p < 0.0001) 


Trend still decreasing but 
reduced to - 0.3% per 
month p = 0.006 




5: Antidepressants: three 
steps towards safer use 


Reduce antidepressant 
duplication 


3.5% 


Trend increasing 0.2% 
per month 


0.6% p = 0.72 


Trend increasing at a rate of 
0.5% per month p = 0.05 


No effect 


(n = 42196) 


Reduce antidepressants 
with tramadol 


5.8% 


Trend decreasing - 
0.6% per month 


-1.0% p = 0.64 


Trend still decreasing 
(non-significant to prior) - 0.8% 
per month p = 0.23 




6: Inhaled respiratory medicines: 
optimising use (n = 28670) 


Reduce multiple device 
use (3 different devices) 


9.5% 


Trend increasing 0.2% 
per month 


-0.11% p = 0.32 


Trend now decreasing 
(non-significant to prior) - 0.2% 
per month p = 0.23 


100 patients no longer on 
4 or more different devices 




Reduce multiple device use 
(4 or more different devices) 


2% 


Trend increasing 0.1% 
per month 


2% p = 0.60 


Trend now decreasing - 0.6% 
per month p = 0.01 




7: PPIs in GORD: Reduce the 
dose - keep the benefits 
(n = 62460) 


Reduce use of high dose 
proton pump inhibitors 
(measured as increase in 
low-dose use) 


2.5% 


Trend of low dose 
increasing 0.6% per 
month 


14.5% (p< 0.0001) 
Increase in low dose 
use 


Trend of low dose increasing at 
greater rate 0.9% per month 
(p = 0.007) 


780 


8: Reducing adverse drug 
events for your veteran 
patients (n = 32484) 


Reduce use of potentially 
inappropriate medicines 


14.7% 


Trend decreasing - 
0.2% per month 


-0.3% p = 0.12 


Trend still decreasing 
(non-significant to prior) - 0.2% 
per month p = 0.63 


No effect 


12: Antipsychotics in 
dementia (n = 6690) 


Reduce antipsychotic use 


0.54% 


Trend increasing 3.6% 
per month 


-14.3% (p< 0.0001) 


Trend increasing but at a reduced 
rate 0.8% per month (p < 0.0001) 


1725 


14: COPD (n = 18096) 


Reduce multiple device 
use (4 or more different 
devices) 


2% 


Trend increasing 0.1% 
per month 


2% p = 0.68 


Trend now decreasing - 1.1% 
per month p <0.0001 


105 patients no longer on 
4 or more different devices 




Reduce use of nebules 


15.2% 


Trend decreasing - 
0.6% per month 


2% p = 0.07 


Trend decreasing (non-significant 
to prior) - 0.8% per month p = 0.2 





NSAID, non-steroidal anti-inflammatory drug; PPI, proton pump inhibitor; GORD, gastro-oesophageal reflux disease; COPD, chronic obstructive pulmonary disease. 
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12: Antipsychotics in dementia 
Rate of use of risperidone 

14 n 




^—Pre-intervention period: Oct 2005-Sep 2007. Monthly change: 1 .036 (1 .03-1 .04); p<0.0001 . This 
means that use of risperidone was increasing by 3.6% per month pre-intervention. 



^—Intervention period: Oct 2007-Jan 2008. Change in use: 0.86 (0.82-0.89); p<0.0001. This means 
that use of risperidone decreased by 14.3% at the time of the intervention. 



^—Post-intervention period: Feb 2008-Sep 2009. Monthly change (relative to pre-intervention): 0.97 
(0.96-0.98); p<0.0001 . This means that the rate of use was decreasing by 2.7% per month relative 
to the pre intervention trend. 

i.e. overall use continued to increase, but only by 0.9% per month. Overall effect: 1725 fewer 
veterans dispensed risperidone. 



Figure 4 Example of an intervention aiming to reduce medicine use. 



The Veterans' MATES program is a national program, 
targeting all general practitioners in Australia. Thus, the 
program requires sustained engagement of participants 
to maintain its success. The outcomes reported here 
demonstrate a bridging of existing evidence-practice 
gaps in diverse therapeutic areas. The behavioural inter- 
vention, utilising patient-specific prescriber feedback, is 
based on evidence from randomised controlled trials [7]. 
The implementation and evaluation frameworks were 
based on behavioural theories and have enabled imple- 
mentation of evidence in practice. The significance of this 
work lies in its ability to bridge the research-practice 
gap with an implementation and evaluation framework 
that enables sustained engagement and demonstration 
of effectiveness. 

The changes in medication use reported in this paper 
are associated with improvements in health outcomes. 
We have previously shown that PPI use in the Australian 
veteran population was associated with increased use of 
antibiotics (RR 1.23, 95% CI 1.21-1.24) and increased 
risk of hospitalisation for pneumonia (RR 1.16, 95% CI 
1.11-1.22) [21]. We have demonstrated increased risk of 
hospitalisation associated with use of NSAIDs amongst 
veterans with diabetes (IRR 1.31, 95% CI 1.08 - 1.60) 
and veterans dispensed medicines indicative of heart fail- 
ure (IRR 1.34, 95% CI 1.13 - 1.58) [22]. We have also 
demonstrated increased risk of death, stroke, hip frac- 
ture and pneumonia with antipsychotic use in the elderly 



veteran population [23-25]. We also demonstrated that 
beta-blocker use was associated with less hospitalisations 
for heart failure in elderly veterans [26]. The Veterans' 
MATES program was associated with declines in use of 
proton pump inhibitors, reducing NSAID use, reducing 
antipsychotic use and increasing beta-blocker use in 
those with heart failure, all suggesting health outcomes 
for veterans have improved. 

The implementation of the Veterans' MATES program 
includes all the elements identified within the National 
Institute for Clinical Excellence Principles of Best Practice 
for Clinical Audit [27] and is consistent with the behav- 
ioural theories, particularly the Precede- Proceed Model of 
Health Promotion [28] and Social Cognitive Theory [11] 
which identify the need for developing supportive envi- 
ronments, raising awareness, developing knowledge and 
skills, encouraging cognitive processing of the information 
and reinforcing messages over time. Creating an environ- 
ment conducive to change is achieved via stakeholder sup- 
port and involvement, which is an ongoing component of 
the program, achieved through twice yearly meetings of 
the practitioner and veteran reference groups, quarterly 
meetings with the national representative steering com- 
mittee and regular attendance at health professional meet- 
ings. Face-to face meetings are called as required with 
national bodies for selected topics. Topic selection is al- 
ways based on an identified medication-related problem, 
where data analysis has revealed a problem which is 



Table 4 Interventions with combination messages (to reduce and increase medicines use) 



Intervention number 
and topic (number of 
veterans targeted) 


Targeted therapy 


Proportion of 
veterans on 
targeted therapy 
at baseline (%) 


24 month 
pre-intervention trend 


Intervention effect (%) 


20 month post 
intervention trend 


Estimated number with changed 
behaviour as a result of intervention; 
sustained over two years 
post intervention 


10: Constipation: a quality 
of life issue for veteran 
patients (n = 29231) 


Increase use of 
osmotic laxatives 

Decrease use of 
contact laxatives 


1 .5% 
2.5% 


Trend increasing 
0.7% per month 

Trend increasing 
0.2% per month 


1 .8% (p 
3.0% (p = 


= 0.11) 
: 0.0007) 


Trend increasing at greater 
rate 1 .2% per month (p = 0.002) 

Trend still increasing (non-significant 
to prior) 0.2% per month (p = 0.87) 


2047 (825 additional people on 
osmotic laxatives, 410 additional 
people on contact laxatives and 
690 less people on bulk laxatives) 




Increase use of 
bulk laxatives 


1 .9% 


Trend decreasing 
- 0.7% per month 


0.8% (p 


= 0.34) 


Trend now decreasing at greater 
rate - 0.9% per month (p = 0.04) 




13: Clopidogrel 
(n = 16867) 


Increase use 
of clopidogrel 
with aspirin 


1 .5% 


Trend increasing 
0.8% per month 


3.0% (p = 


= 0.018) 


Trend still increasing (non-significant 
to prior) 0.7% per month (p = 0.4) 


1114 (825 now on aspirin with 
clopidogrel, 289 no longer on 
NSAIDs with clopidogrel) 




Reduce concurrent 
use of clopidogrel 
with NSAIDs 


1% 


Trend decreasing 
- 0.3% per month 


-5.3% (p 


= 0.002) 


Trend still increasing (non-significant 
to prior) 0.1% per month (p = 0.3) 
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Table 5 Stakeholder evaluation: stakeholders who found the information "useful or very useful" (GPs and pharmacists) 
or "helpful or very helpful" (veterans) 



Intervention number and topic 


GP feedback: 

useful or very useful* 


Pharmacist feedback: 
useful or very useful* 


Veteran feedback: 
helpful or very helpful* 


2: Beta-blockers, take the next step for heart failure 


69% 


N/A* 


81% 


3: Diabetes triple check 


71% 


92% 


82% 


4: Clinical risk management: NSAIDs 


73% 


95% 


80% 


5: Antidepressants: three steps towards safer use 


81% 


92% 


81% 


6: Inhaled respiratory medicines: optimising use 


80% 


91% 


86% 


7: PPIs in GORD: Reduce the dose - keep the benefits 


81% 


95% 


72% 


8: Reducing adverse drug events for your veteran patients 


84% 


95% 


86% 


10: Constipation: a quality of life issue for veteran patients 


84% 


92% 


72% 


12: Antipsychotics in dementia 


85% 


93% 


N/A* 


13: Clopidogrel 


86% 


98% 


87% 


14: COPD 


79% 


90% 


77% 


15: Osteoporosis 


84% 


92% 


77% 


Average 


80% 


93% 


80% 



# Percent of GPs, veterans or pharmacists who provided feedback. 

*No feedback was sought from pharmacists for intervention 2 or from veterans for intervention 12. 



amenable to intervention by audit and feedback and which 
has the support of the clinical reference group and stake- 
holders. Problems which are not measureable in the data 
are not implemented under this program. An intervention 
plan is developed which includes clear objectives, the 
strategies to meet each objective and process impact and 
outcome indicators for evaluation. This plan is endorsed 
by all groups for each intervention. The material includes 
call to action questions to promote cognitive processing. 
Stakeholder feedback through our reference groups, call 
centre, e-mail comments line and one-page response 
forms identify both barriers and enablers to continue to 
improve the program. The evaluation results are routinely 
fed-back to all reference groups and inform the ongoing 
interventions. Further, the ongoing nature of the program 
enables repeat messages over time. 

While it could be argued that programs such as this 
should be evaluated using more rigorous methods, such 
as randomized controlled designs, we advocate that this 
program represents true implementation science. Previ- 
ous randomized controlled trial data have demonstrated 
the efficacy of audit and feedback. Similarly, randomized 
controlled trial data were available for all Veterans' 
MATES interventions targeting medicine efficacy and ei- 
ther randomised controlled trial or well conducted ob- 
servational evidence is available for all medicine safety 
issues targeted. For this reason, this program sought to 
implement the evidence, and as a consequence used 
health program evaluation methods for the analysis. For 
this reason, we are limited to methods of evaluation 
such as time series or non-equivalent groups. As this 



program was a national program, for the majority of in- 
terventions no comparison group was able to be se- 
lected, thus time-series was the method used. A further 
challenge in evaluating ongoing programs are the occur- 
rence of simultaneous interventions, or interventions 
immediately prior or after which may impact on the eva- 
lution and lead to over-estimates or under-estimates of 
results. This may occur due to changes in co-payments, 
product withdrawals, changes to subsidy and changes in 
safety or efficacy or indications. While at times these en- 
vironmental changes may obscure intervention effects, 
well targeted programs can take advantage of these op- 
portunities to enhance intervention effect, for example 
by reinforcing safety messages and giving clear advice on 
what to use instead. The consistency of our results, in 
terms of relative effect size, across a range of topics and 
the consistency of that effect with existing studies using 
controlled designs [7] suggests our evaluation is likely to 
be estimating true effect. 

We have demonstrated that a well designed program 
using proven techniques from implementation research, 
designed with local characteristics in mind and in consult- 
ation with key stakeholders can be effective. The Australian 
medicines and health environment has been exposed to 
a formal National Medicines Policy for a decade [29]. 
This policy initiative has funded national quality use of 
medicines programs, for example the National Prescribing 
Service, which has been in operation for about 10 years. 
While this environment may predispose both health prac- 
titioners and consumers to be favourable to the types of 
interventions implemented by the Veterans' MATES 
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program, the practice changes reported in this paper are 
over-and-above those that have been achieved through 
these other policy initiatives [30-32]. 

Conclusions 

The Veterans' MATES program has been successful in 
achieving positive, significant changes in medicines and 
health service use over time. Educational interventions 
with a clear, single message tended to be most successful 
in changing practice as were those which aimed to in- 
crease use. The Veterans' MATES program provides a 
model that could be replicated in other settings such as 
a national health service or a health insurance organisa- 
tion where bridging the evidence-practice gap is proving 
a significant challenge. 
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